Personal website of Jamie Simon, a scientist specializing in fundamental theory for deep learning. He runs a research lab at the Redwood Center at UC Berkeley with funding from Imbue and recently completed his PhD under Mike DeWeese. The site serves as a hub for his scientific research, personal blog posts regarding science and life adventures, and custom-made puzzles.
Main topics:
* Deep learning fundamental theory
* Research publications
* Science and lifestyle blog
* Puzzle creation
Main topics:
* Deep learning fundamental theory
* Research publications
* Science and lifestyle blog
* Puzzle creation
A practical pipeline for classifying messy free-text data into meaningful categories using a locally hosted LLM, no labeled training data required.
Learn how to label text without the need for task-specific training data by using zero-shot text classification. This guide explains how pretrained transformer models, such as BART, reframe classification as a reasoning task where labels are treated as natural language statements.
Key topics include:
* The core concept of zero-shot classification and its advantages for rapid prototyping.
* Using the Hugging Face transformers pipeline with the facebook/bart-large-mnli model.
* Implementing multi-label classification for texts belonging to multiple categories.
* Improving accuracy through custom hypothesis template tuning and clear label wording.
Key topics include:
* The core concept of zero-shot classification and its advantages for rapid prototyping.
* Using the Hugging Face transformers pipeline with the facebook/bart-large-mnli model.
* Implementing multi-label classification for texts belonging to multiple categories.
* Improving accuracy through custom hypothesis template tuning and clear label wording.
A comprehensive curated collection of Large Language Model (LLM) architecture figures and technical fact sheets. This gallery provides a visual and data-driven overview of modern model designs, ranging from classic dense architectures like GPT-2 to advanced sparse Mixture-of-Experts (MoE) systems and hybrid attention models. Users can explore detailed specifications including parameter scales, context windows, attention mechanisms, and intelligence indices for various prominent models.
Key features include:
* Detailed architecture fact sheets for a wide array of models such as Llama, DeepSeek, Qwen, Gemma, and Mistral.
* An architecture diff tool to compare two different model designs side-by-side.
* Comparative analysis across dense, MoE, MLA, and hybrid decoder families.
* Links to original source articles and technical reports for deeper research.
Key features include:
* Detailed architecture fact sheets for a wide array of models such as Llama, DeepSeek, Qwen, Gemma, and Mistral.
* An architecture diff tool to compare two different model designs side-by-side.
* Comparative analysis across dense, MoE, MLA, and hybrid decoder families.
* Links to original source articles and technical reports for deeper research.
Simon Willison tests OpenAI's newly released ChatGPT Images 2.0 model using a complex Where's Waldo style prompt involving a raccoon holding a ham radio. By comparing results against previous versions and competitors like Google's Nano Banana, the article evaluates the model's ability to handle high-detail illustrations and specific text elements.
Drawing on Marshall McLuhan’s philosophy, this piece warns that while we build AI tools, those same tools ultimately reshape our creative processes. Designers face the dual risks of "AI sycophancy"—where algorithms validate existing biases—and an "illusion of authority" that prioritizes polished speed over genuine depth. To avoid losing their edge, creators must treat AI as a partner for iteration rather than a replacement for critical thinking and human intuition.
* **The Feedback Loop:** Tools aren't neutral; they actively mold the user's cognitive habits.
* **Sycophancy Risk:** AI can act as a "digital yes-man," reinforcing errors instead of challenging them.
* **Superficiality Trap:** Rapid, high-quality outputs can mask a lack of true accountability or substance.
* **Intentional Agency:** Maintaining human intuition is essential to prevent being shaped by the technology.
* **The Feedback Loop:** Tools aren't neutral; they actively mold the user's cognitive habits.
* **Sycophancy Risk:** AI can act as a "digital yes-man," reinforcing errors instead of challenging them.
* **Superficiality Trap:** Rapid, high-quality outputs can mask a lack of true accountability or substance.
* **Intentional Agency:** Maintaining human intuition is essential to prevent being shaped by the technology.
The article explores how artificial intelligence is poised to disrupt traditional organizational structures by collapsing the translation costs between roles. Rather than just speeding up existing workflows, AI enables a fundamental shift from sequential handoffs—like PM to design to engineering—to highly autonomous, small squads and composable capability atoms. As information routing becomes automated, middle management must pivot toward judgment and coaching, while competitive advantage shifts from execution speed to learning speed.
Key points:
- Hierarchy's true function is information routing rather than just authority.
- AI eliminates the translation bottlenecks between product managers, designers, engineers, and QA.
- Organizational models will shift from relay races to simultaneous squad-based work.
- Departments may decompose into independent, composable capability atoms.
- The competitive moat moves from shipping speed to organizational learning speed.
Key points:
- Hierarchy's true function is information routing rather than just authority.
- AI eliminates the translation bottlenecks between product managers, designers, engineers, and QA.
- Organizational models will shift from relay races to simultaneous squad-based work.
- Departments may decompose into independent, composable capability atoms.
- The competitive moat moves from shipping speed to organizational learning speed.
This paper explores how reinforcement learning agents can use environmental features, termed artifacts, to function as external memory. By formalizing this intuition within a mathematical framework, the authors prove that certain observations can reduce the information required to represent an agent's history. Through experiments with spatial navigation tasks using both Linear Q-learning and Deep Q-Networks (DQN), the study demonstrates that observing paths or landmarks allows agents to achieve higher performance with lower internal computational capacity. Notably, this effect of externalized memory emerges unintentionally through the agent's sensory stream without explicit design for memory usage.
- Formalization of artifacts as observations that encode information about the past.
- The Artifact Reduction Theorem proving environmental artifacts reduce history representation requirements.
- Empirical evidence showing reduced internal capacity needs when spatial paths are visible.
- Observation that externalized memory can emerge implicitly in standard RL agents.
- Implications for agent design, suggesting performance gains may come from environment-agent coevolution rather than just scaling parameters.
- Formalization of artifacts as observations that encode information about the past.
- The Artifact Reduction Theorem proving environmental artifacts reduce history representation requirements.
- Empirical evidence showing reduced internal capacity needs when spatial paths are visible.
- Observation that externalized memory can emerge implicitly in standard RL agents.
- Implications for agent design, suggesting performance gains may come from environment-agent coevolution rather than just scaling parameters.
>"For us to trust it on certain subjects, researchers in the growing field of interpretability might need to learn how to open the black box of its brain."
As AI shifts from predictable programs to autonomous neural networks, it has become harder for creators to understand how models reach conclusions. This "black box" problem creates risks in high-stakes fields like medicine and national security, where unaccountable decisions can be life-altering. While interpretability research uses tools like sparse autoencoding to peer inside these systems, the process remains experimental and inconsistent. Researchers are racing to build a reliable toolkit to move from mere observation toward true scientific comprehension.
Key Points:
* Evolution of Complexity: AI has moved from rule-based logic to massive neural networks that learn autonomously, making internal processes difficult to trace.
* High Stakes: Opacity limits AI adoption in critical sectors like healthcare, law, and defense.
* Interpretability Challenges: Current methods for explaining model behavior are often unreliable or prone to deception.
* Potential for Discovery: Emerging tools have already begun uncovering scientific insights, such as new biomarkers for diseases.
* A Developing Science: The field is in its infancy, transitioning from trial-and-error toward a structured scientific discipline.
As AI shifts from predictable programs to autonomous neural networks, it has become harder for creators to understand how models reach conclusions. This "black box" problem creates risks in high-stakes fields like medicine and national security, where unaccountable decisions can be life-altering. While interpretability research uses tools like sparse autoencoding to peer inside these systems, the process remains experimental and inconsistent. Researchers are racing to build a reliable toolkit to move from mere observation toward true scientific comprehension.
Key Points:
* Evolution of Complexity: AI has moved from rule-based logic to massive neural networks that learn autonomously, making internal processes difficult to trace.
* High Stakes: Opacity limits AI adoption in critical sectors like healthcare, law, and defense.
* Interpretability Challenges: Current methods for explaining model behavior are often unreliable or prone to deception.
* Potential for Discovery: Emerging tools have already begun uncovering scientific insights, such as new biomarkers for diseases.
* A Developing Science: The field is in its infancy, transitioning from trial-and-error toward a structured scientific discipline.
BrowSDR is a high-performance, browser-based Software Defined Radio (SDR) receiver designed specifically for HackRF devices. By utilizing WebUSB and WebAssembly, it allows users to tune into various radio modes directly within a web browser without the need for native drivers or software installation. The platform features a multi-VFO architecture for simultaneous frequency monitoring, real-time WebGL waterfall displays, and AI-powered live transcription of demodulated audio.
Key features include:
- Multi-VFO support for independent tuning and DSP settings
- High-speed signal processing via Rust and WASM
- Real-time spectrum analysis and GPU-accelerated waterfall display
- Wide demodulation support including WFM, NFM, AM, USB, LSB, DSB, CW, and raw IQ
- Built-in POCSAG decoding and RDS station information retrieval
- Remote access capabilities via WebRTC/PeerJS
Key features include:
- Multi-VFO support for independent tuning and DSP settings
- High-speed signal processing via Rust and WASM
- Real-time spectrum analysis and GPU-accelerated waterfall display
- Wide demodulation support including WFM, NFM, AM, USB, LSB, DSB, CW, and raw IQ
- Built-in POCSAG decoding and RDS station information retrieval
- Remote access capabilities via WebRTC/PeerJS
